Echo State networks and Neural network Ensembles to predict Sunspots activity

Echo state networks (ESN) and ensembles of neural networks are developed for the prediction of the monthly sunspots series. Through numerical evaluation on this benchmark data set it has been shown that the feedback ESN models outperform feedforward MLP. Furthermore, it is shown that median fusion lead to robust predictors, and even can improve the prediction accuracy of the best individual predictors. 1 Echo State Networks The echo state network (ESN) is a recurrent neural network model trained using supervised learning [1, 2, 3]. In the following we present a brief introduction to the ESN architecture and ESN learning. The ESN Network Model Each neuron , or unit, of the network has an activation state at a given time step n. The network consists of a set of K input units with an activation vector u(n), a set of N inner units with an activation vector x(n), and a set of L output units with an activation vector y(n) [3]. The network has a N × K input connection weight matrix W, a N ×N internal connectivity matrix W, a L × (K +N) output weight matrix W, and optional N × L output global feedback connection weight matrix W. The activation states of the inner units are updated using: x(n+ 1) = f(Wu(n+ 1) + Wx(n) + Wy(n)) (1) where f is the transfer (activation) function of the inner units. The output is calculated using: y(n+ 1) = f(W(u(n+ 1),x(n+ 1),y(n))) (2) where f is the output activation function and u(n+ 1),x(n+ 1),y(n) is the concatenation of the input, inner and output activation vectors. The hyperbolic tangent function (tanh) is usually used with f and f, though other sigmoidal and linear functions can be used as well [3]. ESANN'2009 proceedings, European Symposium on Artificial Neural Networks Advances in Computational Intelligence and Learning. Bruges (Belgium), 22-24 April 2009, d-side publi., ISBN 2-930307-09-9.